This invention relates to a method and apparatus for synchronization of audio and/or video components in a compressed audio/video (A/V) signal receiver.
MPEG is a compressed video signal protocol established by the Moving Pictures Experts Group of the International Standardization Organization. This protocol defines a versatile signal format which includes both intraframe coding and motion compensated predictive coding. Due to variations in coding format frame to frame, and variations in image content, different frames have widely divergent quantities of compressed data. As a consequence of different frames being compressed with different quantities of data etc., frames of data tend to be transmitted asynchronously.
Audio signals may also be compressed according to an MPEG protocol. Compressed audio may be associated with video but transmitted independently. For transmission the compressed audio is segmented into packets which are then time division multiplexed with the compressed video signal in a non synchronous manner.
Associated compressed audio and video components are not only independently asynchronous, but their mutual temporal relationships or synchronism is nonexistent in transmission.
MPEG compressed audio and video component signals may include presentation time stamps (PTS) to establish a reference between particular compressed signal segments and a system reference clock signal. The audio and video PTS""s are utilized by receiver apparatus to both resynchronize respective decompressed components and to restore their temporal interrelationship.
An MPEG or MPEG like A/V receiver will provide the A/V component timing reference signals (PTS""s) coincident with reproduction of associated decompressed component signals. Synchronization apparatus within the receiver will use the PTS""s occurring in the audio and video components to maintain the audio and video components in synchronism. While the system is being brought into synchronization, and the audio is not lip-synced with video, the audio is typically muted because it is perceived that viewers would rather hear no audio, than audio which is not synchronous with the video, at least for short intervals. Muting will generally occur for only relatively short intervals because there are relatively stringent requirements regarding the timing of the transmitted audio and video components for a given standard. Thus synchronization is guaranteed to occur in a short interval.
In receivers designed to decode standardized A/V components a problem exists if the receiver receives a non standard signal, or a standard signal, which has been demultiplexed and remultiplexed in a plurality of virtual transmission channels. In both cases, the relative audio and video component timing relationships may be variable. Lip-sync may not, and muting may occur for long intervals. This situation is untenable.
A second problem exists regarding the method of synchronization when applied to non standard signals. The synchronization process typically includes a coarse mode and a fine mode. Audio data is normally decoded ahead of video data and stored in memory. The stored audio is reproduced from memory when the appropriate video is decoded. However, if the audio in the nonstandard signal arrives too late, there will be no audio ready to reproduce and synchronization cannot occur. On the other hand, if the audio occurs too early, (relative to buffer size) there will be insufficient memory to store the audio and data will be lost. Again synchronization cannot occur, possibly for long intervals.
It will be appreciated that the viewer will prefer that both video and audio be reproduced in these instances, rather than suffer long periods of audio muting. Thus, there is a need for an adaptive synchronization system in digital A/V receivers to accommodate both standard and non standard received signals.
An A/V receiver according to the present invention provides A/V component timing reference signals (PTS""s) coincident with the reproduction of associated decompressed component signals. Synchronization circuitry generates a function of the difference of occurring component audio and video PTS""s. This function is indicative of relative audio and video synchronization. If the value of the function is within a certain range, the synchronization process is continued. If the value of the function exceeds a predetermined level, the synchronization process is terminated and non synchronized audio and video components are reproduced. In a further embodiment, for a given range of values of the function, the synchronization process continues, but the audio component is muted.